The present invention relates to an image processing method and apparatus, and in particular those for use in video devices such as television sets and video tape recorders, information processing systems such as personal computers, and printing-related devices such as image scanners and printers.
More particularly, the invention relates to an image conversion process for modifying the tone characteristic of the video data or image data, data processing for converting image data represented by red, green and blue to printing data of yellow, magenta and cyan, with or without black, data processing for converting sensor data into color-separated data of red, green and blue.
A prior art image information processing system, which has been used in display systems in information processing systems such as computers, will first be described.
FIG. 61 is an example of a conventional tone processing device shown in Japanese Patent Kokoku Publication No. S57-9072. It comprises a refresh pattern memory 100 storing image information in digital values, a table select circuit 101, a RAM (random-access memory) 102 for storing tone compensation information, and a controller 103 for controlling the table select circuit 101 and the RAM 102.
The RAM 102 includes a plurality of compensation tables, each of which stores compensation data. The refresh pattern memory 100 stores image data X consisting of a plurality of pixel data forming one screen or raster. The image data X are sent as image density information from a host computer (not shown). The image data X from the refresh pattern memory 100 are converted at the table select circuit 101 having a function of selecting the compensation characteristic, into synthetic address data, which is then converted at a selected one of the compensation tables in the RAM 102. The output of the RAM 102 is converted data Y corresponding to the input image data X.
The controller 103 generates conversion characteristic selection data and the like, and controls the timings of the table select circuit 101 and the RAM 102.
Usually, the image data include 8 to 10 bits for expressing the density of each pixel. To implement the table conversion for one compensation characteristic, the capacity of the memory required is 256 B (B being an abbreviation for byte(s)) for the case of 8 bits, and 1 kB for the case of 10 bits. The refresh pattern memory 100 is therefore configured of about 1200 gates for the case of 8 bits, and about 3000 gates for the case of 10 bits. If ROMs are used in place of RAMs, the numbers of the gates are reduced to 400 and 700, respectively. However, with ROMs, it is not possible to alter the compensation characteristic depending on the image information.
The term "image data" as used herein include not only the image data for expressing stationary picture, but also video data for expressing moving picture.
Other examples of prior art for tone conversion are disclosed in Japanese Patent Kokoku Publication H3-81346, Japanese Patent Kokoku Publication No. 114-41551, and Japanese Patent Kokoku Publication No. S57-9072.
FIG. 62 shows another example of a conventional tone conversion device. As illustrated, it comprises a density table 104 to which a luminance signal Y obtained by reading a color original is input, a density compensation table 105, logarithm converters 106a to 106c to which R, G and B color-separated data are input, and adders 107a to 107c.
A table of Correlation between the luminance signal Y and the original density is prepared experimentally in advance, and this table is stored in the density table 104. By applying the luminance signal Y to the density table 104, density data Din of the original are obtained. The density data Din are input to the density compensation table 105. The output of the density compensation table 105 is a compensation amount .alpha. corresponding to the density data Din.
The color-separated data (R, G, B) having been obtained by scanning the original are converted at the logarithmic converters 106a to 106c to density data Dr, Dg and Db, which are added to the compensation amount .alpha. at the adders 107a to 107c. The output of the adders 107a to 107c are the density compensation data DR, DG and DB.
Each of the image data of red, green and blue, are represented by 8 to 10 bits. To implement the table conversion of one density compensation characteristic, the density compensation table 105 needs to have a capacity of 256 B to 1 kB.
The image data obtained from color original needs to be subjected to tone conversion as well as color conversion to optimize the printing. This is to avoid degradation of the picture quality which may results from the ink spectrum color-impurity due to the fact that the inks used for printing are not of saturated colors, and non-linearity of the transfer characteristic of the printing. The color conversion is to compensate the degradation in the picture quality, and to provide the printer with printing image having a good color reproducibility.
Two methods have been developed for the color conversion, namely matrix calculation method and table conversion method. In the matrix calculation method, the following expression (27) is used as a basic formula. ##EQU1##
In the above formula, i=1 to 3, j=1 to 3, Y, M and C represent printing data, R, G and B represent image data, and (Aij) represents a color conversion coefficient matrix.
The linear calculation of the formula (27) is not sufficient to produce good conversion characteristic to Compensate the non-linearity in the transfer characteristic of the printing with regard to printing data.
A method for improving the color conversion characteristic is disclosed in Japanese Patent Kokoku Publication H2-30226, in which the following matrix formula is adopted. ##EQU2##
In the above formula, i=1 to 3, j=1 to 10, Y, M and C represent printing data, R, G and B represent image data, N represents a constant, and (Dij) represents a color conversion coefficient matrix.
In the matrix calculation of the formula (28), the image data in which achromatic components and color components are both contained is directly used, so that interference occurs in the calculation. That is, changing the color conversion coefficients (matrix operator) with respect to one of the components or hues also affects other components or hues. It was therefore difficult to compensate the degradation in the picture quality due to the color-impurity of the inks and to achieve a good conversion characteristic.
A solution to this problem is disclosed in Japanese patent Kokai Publication H1-47174. FIG. 63 shows an example of the color conversion device of this prior art. As illustrated, it comprises a minimum value circuit 108, a subtractor 109, a coefficient generator 110, a matrix calculator 300, a ROM 112 and a synthesizer 113.
Input to the minimum value circuit 108 are image data R, G and B (representing the red, green and blue components of the pixel in question). The minimum value circuit 108 produces the minimum value, .alpha., of the three inputs. The subtractor 109 determines the differences between the image data R, G and B, and the minimum value .alpha., to produce difference data R1, G1 and B1. The minimum value .alpha. corresponds to the achromatic component of the image data, and the difference data R1, G1 and B1 correspond to the color components, respectively. Input to the matrix calculator 300 are the difference data R1, G1 and B1, as well as the coefficients generated by the coefficient generator 110, and a matrix calculation given by the formula (28) is performed to produce the color ink data Ya, Ma and Ca. The ROM 112 contains a table for converting the minimum value .alpha. to produce achromatic data Yb, Mb and Cb. The synthesizer 113 determines the sums of the color components for the respective inks and the achromatic components to produce Ya+Yb, Ma+Mb, and Ca+Cb, respectively, as the printing data Y, M and C for yellow, magenta and cyan.
According to this color conversion, the interference between the color components and the achromatic component can be removed. However, the interference between the hues within each color component cannot be removed.
According to the color conversion method using the conversion tables, as shown in FIG. 61 and FIG. 62, the red, green and blue image data are input to a conversion table, and printing data Y, M and C stored in a memory, such as a ROM. With this method, any desired conversion characteristic can be obtained, and a color conversion with excellent color reproducibility can be realized.
However, in the simple configuration in which for each of the combinations of the image data R, G and B, the conversion table must have a capacity of about 400 Mbits. The color conversion device disclosed in Japanese Patent Kokoku Publication H5-73310 shows a method for reducing the memory capacity. However, even with the reduction, the memory has to have a capacity of about 5 Mbits. A shortcoming of this method is therefore a capacity for each of the conversion characteristic is large, and it is difficult to configure the circuit by means of an LSI. Moreover, it is difficult to adopt the device to alteration in the use of the different inks, and to cope with the variation in the conditions of the printing.
Because the prior art image processing (tone conversion or the color conversion) is performed using the conversion tables as described above, the processing is limited to those in accordance with the characteristics stored in the tables. As a result, the characteristics stored in the tables cannot be altered readily or at will. There are on the other hand demands for a diversity of types of processings. They include:
(a) gamma characteristic processing inherent to video equipment; PA1 (b) matching (unification) of the tone characteristic between video equipment, image equipment and printing equipment; and PA1 (c) tone processing matching the contrast characteristic of the image, or the luminosity characteristic or the preference of the individual viewer. PA1 setting a functional formula including at least two of a first-order term of the input image data X, a second-order term of the input image data X, a third-order term of the input image data X, and a fraction term of the input image data X; and PA1 outputting the converted data Y in accordance with the functional formula. PA1 setting a functional formula including a logarithmic term of the image data X; and PA1 outputting the converted data Y in accordance with the functional formula. PA1 (a) generating complementary color data Ci, Mi and Yi from the image data R, G and B; PA1 (b) determining EL minimum value .alpha. and a maximum value .beta. of the complementary color data in accordance with the following expression: PA1 (c) generating hue data r, g, b, y, m and c from the complementary color data, the minimum value and the maximum value, in accordance with the following equations: EQU r=.beta.-Ci EQU g=.beta.-Mi EQU b=.beta.-Yi EQU y=Yi-.alpha. EQU m=Mi-.alpha. EQU c=Ci-.alpha. PA1 (d) generating predetermined matrix coefficients (Eij) and (Fij), with i=1 to 3 and j=1 to 3 for (EiJ) and i=1 to 3 and j=1 to 12 for (Fij); and PA1 (e) determining printing data C, A, Y in accordance with a matrix calculation of the following formula (15): ##EQU3## PA1 (a) generating complementary color data Ci, Mi and Yi from the image data R, C and B; PA1 (b) determining a minimum value .alpha. and a maximum value .beta. of the complementary color data in accordance with the following expression: PA1 (c) generating hue data r, g, b, y, m and c from the complementary color data, the minimum value and the maximum value, in accordance with the following equations: EQU r=.beta.-Ci EQU g=.beta.-Mi EQU b=.beta.-Yi EQU y=Yi-.alpha. EQU m=Mi-.alpha. EQU c=Ci-.alpha. PA1 (d) generating predetermined matrix coefficients (Eij) and (Fij), with i=1 to 3 and j=1 to 3 for (Eij) and i=1 to 3 and j=1 to 14 for (Fij); and PA1 (e) determining printing data C, M, Y in accordance with matrix calculation of the following formula (18): ##EQU6## PA1 (a) generating complementary color data Ci, Mi and Yi from the image data R, G, and B; PA1 (b) determining a minimum value .alpha. and a maximum value .beta. of the complementary color data in accordance with the following expression: PA1 (c) generating hue data r, g, b, y, m and c from the complementary color data, the minimum value and the maximum value, in accordance with the following equations: EQU r=.beta.-Ci EQU g=.beta.-Mi EQU b=.beta.-Yi EQU y=Yi-.alpha. EQU m=Mi-.alpha. EQU c=Ci-.alpha. PA1 (d) dividing the minimum value into the printing data K and a remainder data .alpha.-K; PA1 (e) generating predetermined matrix coefficients (Eij) and (Fij), with i=1 to 3 and j=1 to 3 for (Eij) and i=1 to 3 and j=1 to 12 for (Fij); and PA1 (f) determining printing data C, M, Y in accordance with matrix calculation of the following formula (21): ##EQU9## PA1 (a) generating complementary color data Ci, Mi and Yi from the image data R. G and B; PA1 (b) determining a minimum value .alpha. and a maximum value .beta. of the complementary color data in accordance with the following expression: PA1 (c) generating hue data r, g, b, y, m and c from the complementary color data, the minimum value and the maximum value, in accordance with the following equations: EQU r=.beta.-Ci EQU g=.beta.-Mi EQU b=.beta.-Yi EQU y=Yi-.alpha. EQU m=Mi-.alpha. EQU c=Ci-.alpha. PA1 (d) generating predetermined matrix coefficients (Eij) and (Eij), With i=1 to 3 and j=1 to 3 for (Eij) and i=1 to 3 and j=1 to 14 for (Fij); and PA1 (e) determining Sprinting data C, M, Y in accordance with matrix calculation of the following formula (24): ##EQU12## PA1 (a) means for selecting image data and address data; PA1 (b) means for switching the direction of transfer of write data and read data; PA1 (c) rewritable memory means; PA1 (d) means for generating the write data by means of functional calculation; PA1 (e) means for generating the address data; and PA1 (f) means for controlling the operation of the means (a) to (e). PA1 (a) means for selecting three-color conversion or four-color conversion, with regard to the conversion into the printing data; PA1 (b) means for selecting use or non-use of muddiness-removal function in the color conversion; PA1 (c) means for selecting use or non-use of fine adjustment of achromatic components; PA1 (d) means for selecting constants corresponding to division function; and PA1 (g) means for selecting calculation constants corresponding to a set of plurality of inks. PA1 (a) means for converting the image data R, G and B expressed by three colors red, green and blue into the printing data; PA1 (b) means for converting the image data R, G and B expressed by three colors red, green and blue into the display data; PA1 (c) means for converting the image data from a sensor into color-separated data, or image data R, G and B expressed by throe colors red, green and blue; PA1 (d) means for converting a first type of image data into a second type of image data; PA1 (e) means for converting a first type of printing data into a second type of printing data; PA1 (f) means for converting a first type of color-separated data into a second type of color-separated data; and PA1 (g) means for performing conversion in such a manner as to unify or obtaining matching of the color reproducibility in each of the combinations of at least three of the color-separated data, image data, printing data and display data. PA1 means for selecting one of the processing characteristics, according to the characteristics of an input equipment, or an output equipment; and PA1 means for varying and setting a conversion function or constants in the conversion function according to the selected processing characteristics.
If it is attempted to meet the above demands by increasing the capacity of the memory, the total capacity of the memory is too large to compose the memory of LSIs.
Moreover, it is difficult to implement the equivalent functions by means of software.
Similarly, in the prior art color conversion using the matrix calculation, compensation (retouching) cannot be performed independently for each hue. As a result, calculation interference occurs between different hues. It is therefore difficult to properly set color conversion coefficients, and to realize satisfactory conversion characteristics for all colors.
The color conversion using conversion tables encounters the problems similar to those which the tone compensation encounters; that is, the problem that the capacity of the memory required is intolerably large, and the conversion characteristic cannot be altered flexibly.